One of the primary uses of computers is for information storage and retrieval. Database systems have been developed that allow a computer to store a large amount of information in a way that allows a user to search for specific information in the database and retrieve such specific information. For example, an insurance company may have a database that includes all of its policy holders and their current account information, including payment history, premium amount, policy number, policy type, exclusions to coverage, etc. A database system allows the insurance company to retrieve the account information for one or more policy holders among thousands, and perhaps millions, of policy holders in its database and generate reports based upon specific user parameters. Applications have been developed that search databases for specified keywords and return a list of results where the keywords are found. Such applications are generally referred to as search engines. Frequently, users need to conduct searches in order to retrieve data spread over several databases, sometimes spread across multiple countries, and managed by multiple search engines. Searches conducted using web search engines (such as Google by Google Inc. of Mountain View, Calif., US) to search for information on the World Wide Web are good examples of such searches. Such extensive and complicated searches are typically conducted by meta search engines.
A meta search engine is a search engine that sends user requests to several other search engines and/or databases and aggregates the results into a single list or displays them according to their source. Meta search engines enable users to enter search criteria once and access several search engines simultaneously. Meta search engines operate on the premise that the World Wide Web is too large for any one search engine to index it all and that more comprehensive search results can be obtained by combining the results from several search engines. This also may save the user from having to use multiple search engines separately.
The performance of meta searches mainly depends on the reaction time of the underlying search engines and on the overhead of distributing queries to all search engines. Distributing all queries to all search engines leads to poor performance from the end user perspective, especially if the meta search uses many search engines. Another performance bottleneck is usually the wide variety of search engines used for conducting meta searches such as the search engines used for an enterprise search spanning the whole field of search on structured data (e.g. business objects) and unstructured data (documents).
One solution that has been employed to overcome the above performance bottlenecks is the use of pre-configured keywords that address one or a group of search engines. Thus a query for conducting a search based on a pre-configured keyword would be forwarded only to those search engines that have already been configured for the keyword. Although this method results in some performance improvements it has many limitations. A search conducted using a pre-configured keyword is limited to the configured search engines only. In addition, the keywords are typically language dependent and thus create an overhead for internationalization.
Thus there is no meta search technique currently available that intelligently targets search engines in a network and at the same time provides a comprehensive search. An intelligent and comprehensive search targets every search engine that may possibly contain data matching a provided keyword.